Concepts in Conceptual Clustering

Although it has a relatively short history, conceptual clustering is an especially active area of research in machine learning. There are a variety of ways in which conceptual patterns (the Al contribution to clustering) play a role in the clustering process. Two distinct conceptual clustering paradigms (conceptual sorting of exemplars and concept discovery) are described briefly. Then six types of conceptual clustering algorithms are characterized, attempting to cover the present spectrum of mechanisms used to conceptualize the clustering process. I CONCEPTUAK CKUSTERING: The New Frontier Ever since Michalski wrote about conceptual clustering as a new branch of machine learning (Michalski 1980) there has been ever increasing attention to that family of machine learning tasks. Several researchers have been involved in conceptual clustering research, though early research (the next two citations in particular) was not conducted in the name of conceptual clustering. Wolff (1980) describes MK10. an agglomerative hierarchical data compression system that is able to generate conjunctive descriptions of clusters based on co-occurrences of feature values Kebowitz (1982 and 1983) describes UNIMEM and IPP systems that use what he calls Generalization Based Memory to incrementally clump exemplars into overlapping conceptual categories based on predictive features. Michalski and Stepp (1983) describe CKUSTER/2. a conceptual clustering algorithm for building polythetic clusterings (clusterings whose differences depend on discovered conjunctive concepts rather than variations in the value taken by a single attribute). Kangley and Sage (1984) describe DISCON. an ID3-like (Quinlan 1983) optimal classification tree builder that forms monothetic hierarchical clusterings given a list of "interesting" attributes. Fisher (1984) describes RUMMAGE, a DISCON-like program that does some generalization over attribute values and uses non-exhaustive search. Stepp (1984) describes CKUSTER/S. a conjunctive conceptual clustering algorithm for use on structured exemplars. Kangley, Zytkow. Simon, and Bradshaw (1985) describe GIAUBER, a concept discovery system based partly on MK10. that employs conceptual clumping based on most commonly occurring relations in data. Stepp and Michalski (1986) describe algorithms for incorporating background knowledge and classification goals. Mogensen (1987) describes CKUSTER/CA. a program that forms clusters of structured objects in a goal-directed way through the use of Goal Dependency Networks. Taken together, there is a large diversity of algorithms that now are described by the term conceptual clustering. Fisher and Kangley (1985) provide two views of conceptual clustering (as extended numerical taxonomy, and as concept formation) and This research was supported in part by the National Science Foundation under grant NSF 1ST 85-11170. also give an enlightened characterization of several conceptual clustering algorithms. In the following sections, two somewhat different views of conceptual clustering are described. The first view is that of cluster formation per se. whose goal is the determination of extensionally defined clusters. The conceptual part of the process lies in how the exemplars are agglomerated/divided rather than in how the clusters are described (i.e.. the cluster forming mechanism need not maintain any cluster descriptions). The second view is that of concept formation, with exemplars as the catalyst. Under this view clusters are formed according to their conceptual descriptions, i.e., the system must constantly maintain conceptual descriptions of clusters and cluster membership is constrained by the concepts available to describe the results. Following the terminology of psychology, the first view will here be called conceptual sorting. The second view wil l be called concept discovery. Each in its own way can be said to involve conceptual clustering. II CONCEPTUAK CKUSTERING AS CONCEPT SORTING The process of clustering is to group exemplars in some interesting way (or ways) such as a hierarchy of categories or a tree structure (dendrogram). Numerical taxonomy readily provides such groupings, but the groups have little or no conceptual interpretation One view of conceptual clustering proposes to produce interesting groupings and then provide them with a conceptual interpretation. That is. to build extensionally defined categories (by enumerating their members) and then find a conceptual interpretation. Naturally, some subpopulations of exemplars are easier to interpret (i.e.. form better conceptual clusters) than others. Fisher (1985) proposes such a view, and states that the two phases (called the aggregation and characterization problems, respectively) are not independent. That the clustering and characterization phases are not independent (assuming they are separate processes) is precisely one of the facets that distinguishes conceptual clustering from "regular" clustering. Indeed, one can perform statistical clustering, take the extensionally defined resulting clusters and then generate conceptual interpretations for them. There are cluster ing problems for which this is an acceptable approach—cluster analysis was done exclusively just this way for a long while, with the analyst doing all the interpretation. But in general, concepts derived from independently rendered clusters have potentially messy conceptual characterizations, involving disjunctive conceptual forms (Michalski and Stepp 1983) But one should note that certain patterns of disjunction can be restated as polymorphic concepts ("n of m properties must be present") and some clustering research is directed at finding polymorphic classifications (e.g.. (Hanson and Bauer 1986)). A major reason independently rendered clusters can have rather unappealing conceptual interpretations is that they